Skip to content

chore(deps): bump torch from 2.4.1 to 2.8.0 in /app#1

Closed
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/app/torch-2.8.0
Closed

chore(deps): bump torch from 2.4.1 to 2.8.0 in /app#1
dependabot[bot] wants to merge 1 commit into
mainfrom
dependabot/pip/app/torch-2.8.0

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Apr 18, 2026

Copy link
Copy Markdown

Bumps torch from 2.4.1 to 2.8.0.

Release notes

Sourced from torch's releases.

PyTorch 2.8.0 Release Notes

Highlights

... (truncated)

Commits
  • ba56102 Cherrypick: Add the RunLLM widget to the website (#159592)
  • c525a02 [dynamo, docs] cherry pick torch.compile programming model docs into 2.8 (#15...
  • a1cb3cc [Release Only] Remove nvshmem from list of preload libraries (#158925)
  • c76b235 Move out super large one off foreach_copy test (#158880)
  • 20a0e22 Revert "[Dynamo] Allow inlining into AO quantization modules (#152934)" (#158...
  • 9167ac8 [MPS] Switch Cholesky decomp to column wise (#158237)
  • 5534685 [MPS] Reimplement tri[ul] as Metal shaders (#158867)
  • d19e08d Cherry pick PR 158746 (#158801)
  • a6c044a [cherry-pick] Unify torch.tensor and torch.ops.aten.scalar_tensor behavior (#...
  • 620ebd0 [Dynamo] Use proper sources for constructing dataclass defaults (#158689)
  • Additional commits viewable in compare view

@dependabot dependabot Bot added dependencies Pull requests that update a dependency file python Pull requests that update python code labels Apr 18, 2026
@github-actions

Copy link
Copy Markdown

👍 @dependabot[bot]

Thank you for raising your pull request and contributing to voscript.
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为 voscript 做出贡献,请确保您已经遵循了我们的贡献指南,我们会尽快审查它。
如果您遇到任何问题,请随时与我们联系。

@dependabot dependabot Bot force-pushed the dependabot/pip/app/torch-2.8.0 branch from 0961fe2 to f26ef53 Compare April 18, 2026 15:52
Bumps [torch](https://github.com/pytorch/pytorch) from 2.4.1 to 2.8.0.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](pytorch/pytorch@v2.4.1...v2.8.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.8.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot Bot force-pushed the dependabot/pip/app/torch-2.8.0 branch from f26ef53 to 6e43454 Compare April 19, 2026 14:40
MapleEve added a commit that referenced this pull request Apr 21, 2026
* feat: add overlapped speech detection (OSD) + MossFormer2 separation

Method A — OSD only (pyannote OverlappedSpeechDetection):
- TranscriptionPipeline.detect_overlaps(audio_path, onset) → {intervals, total_s, overlap_s, ratio, count}
- TranscriptionPipeline.osd_pipeline lazy property (onset-cache-aware)
- /api/transcribe accepts osd=true, osd_onset=0.08
- Each segment gets has_overlap: bool
- Result gets overlap_stats: {ratio, overlap_s, total_s, count}

Method B — OSD + MossFormer2 (clearvoice):
- TranscriptionPipeline.separate_overlaps(audio_path, n_speakers=2) → list[str]
- /api/transcribe accepts separate_speech=true (only runs when osd=true and ratio>0)
- Result gets separated_tracks: [{track, segments}]

Frontend:
- OSD checkbox in upload options
- ⚡ 重叠 amber badge on overlapping segments
- Overlap stats summary banner above segment list

* Revert "feat: add overlapped speech detection (OSD) + MossFormer2 separation"

This reverts commit ac4db1c.

* fix(pipeline): MIN_EMBED_DURATION 改为1.5s,支持env配置

- [CQ-H1] WeSpeaker ResNet34 推荐 ≥1.5s 输入;提高最短 turn 阈值并新增 MIN_EMBED_DURATION / MAX_EMBED_DURATION env 支持
- 超过 MAX_EMBED_DURATION 的 chunk 自动截断,避免显存浪费
- [BP-H6] use_auth_token 已废弃,改为 token(huggingface_hub >= 0.17)
- [BP-L3] 整理 Path import 到文件顶部,保留 GPU 库的 lazy import

* fix(security): 修复 XSS - escapeJs 完整转义 + data-* 事件委托

- 删除不完整的 escapeJs(仅转义 '/\),统一使用 escapeHtml 覆盖 &"'<>
- 动态生成的按钮改用 data-action/data-* 属性 + 全局事件委托,
  消除用户数据拼入 inline onclick 字符串的 XSS 通道
  (enrollSpeaker / deleteVp / renameVp / loadTranscription)
- 新增 CSP meta 标签作为服务端响应头的 fallback 防线

* fix(dockerfile): 删除冗余chown,添加digest pin TODO

- BP-L4: 删除 COPY --chown 之后冗余的 RUN chown -R app:app /app
- CD-H6 / BP-M6: 为基础镜像添加 digest pin TODO 注释
- BP-L5: 为 git 依赖添加 fake_git hack 清理后可移除的 TODO

* fix(security): ffmpeg 添加超时保护,删除 /tmp/_fake_git 死代码

- _convert_to_wav 增加 FFMPEG_TIMEOUT_SEC(默认 1800s)超时,
  防止畸形音频使 ffmpeg 死循环占满 CPU/内存、挂起 GPU 信号量;
  TimeoutExpired 时清理残留 wav 并返回 504
- 删除 _load_deepfilternet 中 /tmp/_fake_git shim 与 PATH prepend,
  容器 Dockerfile 已安装真实 git,该 world-writable 回退路径为
  死代码且构成 CWE-427 不可信搜索路径风险

* fix(voiceprint): identify()零向量防御,use_auth_token改token

- [CQ-H9] identify() 入口拒绝零向量 query,返回 (None, None, 0.0),避免 AS-norm 与 raw cosine 语义不一致
- [BP-H9] update_speaker 动态 SQL f-string 拆为两条静态 SQL(有/无 name 两路径)
- [BP-M7] Optional[T] 类型注解全量改为 T | None(Python 3.10+ 风格)
- [CQ-M12] build_cohort_from_transcriptions 硬编码 shape==(256,) 改为 EMBEDDING_DIM env 可配
- [CQ-M3] build_cohort 的 except Exception 改为 logger.warning,不再吞异常
- [BP-L3] 函数体内 base64/glob/os import 移到文件顶部

* fix(security): 添加安全响应头中间件,修复日志注入

- 新增 add_security_headers middleware,默认注入
  X-Content-Type-Options / X-Frame-Options / Referrer-Policy /
  X-XSS-Protection,与 CSP meta 配合构成最小安全头集
- 新增 _safe_log_filename 去除控制字符(\\x00-\\x1f/\\x7f),
  在上传路径用它清洗 file.filename,避免换行/ANSI 控制符
  原样写入日志造成 CWE-117 日志注入

* fix(main): reassign_speaker同步speaker_map

- [CQ-H7] reassign_speaker 更新 segment 时同步更新 speaker_map 中对应 label 的 matched_name/matched_id,保证人工纠错传导
- [CQ-H6] 空 embeddings 时写入 result.json 的 warning="no_speakers_detected" 标志,前端可区分"识别失败"与"无可用 speaker"并避免传 'undefined' 字符串

* fix(frontend): 轮询改指数退避,fetch 错误提示

- pollStatus 从固定 2000ms setInterval 改为 setTimeout 指数退避
  (×1.5,封顶 10000ms),减少长任务轮询压力
- loadHistory / loadTranscription / loadVoiceprints / enrollSpeaker /
  renameVp / deleteVp 外层补 try/catch,非 2xx 响应与网络错误通过
  toast 向用户提示状态码,不再静默失败(avoid 401/413 重复提示,
  由 apiFetch 统一处理)

* fix(docs): 修正AS-norm冷启动、阈值、cohort生命周期、OSD删除、enroll幂等性、jobs/transcriptions双状态

- DOC-H1: 修正 AS-norm 冷启动行为描述(cohort=0 / <10 / >=10 三档阈值与路径)
- DOC-H3: 阈值描述改为自适应(单样本~0.70,spread放宽,绝对下限0.60)
- DOC-H5: 新增 AS-norm cohort 生命周期章节(启动构建、任务不刷新、rebuild-cohort 手动触发)
- DOC-H4: 标注 OSD 功能已在 v0.5.x 中移除(changelog/benchmarks 加历史注释)
- DOC-H2: enroll 端点添加幂等性警告(不传 speaker_id 会产生重复记录)
- DOC-H6: /api/jobs 说明进程重启后返回 404,持久化结果改用 /api/transcriptions
- DOC-M1: /api/transcriptions/{id} 补充 speaker_map 与 unique_speakers 字段说明
- DOC-M5: 0.5.0 升级迁移说明,历史转录需 rebuild-cohort 才能激活 AS-norm

* feat(ci): 添加基础CI workflow + docker-compose healthcheck

- CD-C1: 新增 .github/workflows/ci.yml 做基础 lint (ruff) + pytest 守门
  - lint: ruff check + ruff format --check,忽略 E501
  - test: 轻量依赖跑单元测试,排除 e2e/separation;暂用 || true 避开针对已删除
    OSD 功能的测试,待测试套件修复后改为强制失败
- CD-M2: docker-compose.yml 为 voscript 服务加 healthcheck,使用 /healthz 探测,
  start_period=120s 覆盖首次模型加载时间

* fix(style): CORS headers 收敛,__import__ 改标准 import

- threading 提升到模块顶部 import,删除函数体内的
  import threading as _threading 和 __import__("threading") 丑陋用法,
  _gpu_sem / _hash_index_lock 改用 threading.Lock/Semaphore
- CORSMiddleware 的 allow_headers 从通配 ["*"] 收敛为显式白名单:
  Authorization / Content-Type / X-API-Key / X-Request-Id,
  符合最小权限原则

* fix(quality): 静态SQL替换f-string,异常日志改善

- [CQ-M13] _format_srt_time / _format_timestamp 防御 None / NaN / 负秒,避免 int() 抛异常
- [CQ-M3] _run_transcription 中两处 CUDA 缓存 flush 的 except Exception: pass 改为 logger.warning
- [CQ-M10] rebuild-cohort 端点返回 skipped 字段,通过 voiceprint_db.last_cohort_skipped 暴露跳过/损坏文件数

* fix(perf+docs): PERF-H8 batch cosine scan, DOC-C1 similarity semantics, DOC-C2 API_KEY warning, CD-C2 pip-audit CI

* fix(security): SEC-C1 pickle RCE, SEC-C2 path traversal, SEC-C3 API_KEY warning, BP-C2 path param validation, CD-C3 daemon thread

- SEC-C1 (CVSS 9.1): np.load(emb_path) → np.load(emb_path, allow_pickle=False)
  in enroll_speaker to prevent pickle-based RCE via crafted .npy files.

- SEC-C2 (CVSS 9.6): Add _safe_tr_dir(tr_id) and _safe_speaker_label(label)
  validators. All route handlers that touch TRANSCRIPTIONS_DIR / tr_id now
  call _safe_tr_dir(); enroll_speaker validates speaker_label before building
  the embedding filename. Both use allowlist regex + .resolve() confinement.

- SEC-C3 (CVSS 9.8): Add ALLOW_NO_AUTH env var. When API_KEY is None and
  ALLOW_NO_AUTH != "1", emit a highly-visible WARNING so operators cannot
  accidentally ship the service open. Service still starts (warning not fatal).

- BP-C2: All tr_id path parameters now carry an Annotated[str, FPath(pattern=...)]
  constraint so FastAPI validates the format at the framework layer before any
  handler code runs (get_transcription, reassign_speaker, export_transcription).

- CD-C3: Thread(..., daemon=True) on the transcription worker so SIGTERM
  causes the process to exit cleanly rather than hanging on in-flight jobs.

* fix(logic): CQ-C1 cohort auto-rebuild, CQ-C2 AS-norm threshold, CQ-C3 add_speaker dedup, CQ-C4 vec serialize, CQ-H2 LRU jobs, CQ-H5 fcntl lock

CQ-C1: _run_transcription now increments a counter after each successful job
and rebuilds the AS-norm cohort when absent or every 10th transcription, so
normalization activates without a server restart.

CQ-C2: identify() no longer sets asnorm_active=True when ASNormScorer.score()
fell back to raw cosine (cohort < 10). Score is only treated as AS-norm
normalized — and compared against the 0.5 operating threshold — when the
cohort has >= 10 embeddings; otherwise the adaptive per-speaker threshold
logic takes over.

CQ-C3: add_speaker() now performs a case-insensitive name lookup before
INSERT. Duplicate names route to update_speaker() (append sample + recompute
average) instead of creating a second record.

CQ-C4: The sqlite-vec MATCH query now serialises the query vector via
sqlite_vec.serialize_float32() when available, falling back to
struct.pack('<Nf', ...) for explicit little-endian byte order. Replaces
query.tobytes() which is platform byte-order-dependent.

CQ-H2/PERF-C1: The unbounded `jobs: dict` replaced with _LRUJobsDict
(OrderedDict-backed, thread-safe, bounded by JOBS_MAX_CACHE env var,
default 200). Prevents memory growth from accumulating full result payloads
indefinitely.

CQ-H5: _hash_index_lock (threading.Lock, single-process only) replaced with
_with_file_lock() using fcntl.flock(LOCK_EX) for cross-process mutual
exclusion when running uvicorn --workers > 1. Falls back to thread lock on
non-Unix platforms.

* perf(pipeline): PERF-H1 segment-level torchaudio.load, avoid whole-file OOM

Replace the single torchaudio.load(audio_path) call that decoded the
entire WAV file into memory with per-segment loads using the
frame_offset/num_frames API (torchaudio >= 0.9).  File metadata is
obtained via torchaudio.info() which reads only the header.  Each
iteration allocates at most MAX_EMBED_DURATION * sr * 4 bytes (~640 KB
at 16 kHz / 10 s) instead of the full file, eliminating OOM on 2-hour
recordings (~900 MB–2 GB WAV).  Resampling and mono downmix are applied
per chunk; a try/except guard logs and skips unreadable segments without
aborting the pipeline.

* refactor(arch): extract config.py

Centralise all os.getenv() calls into app/config.py so environment-variable
names are defined in one auditable place. Removes scattered getenv() calls
from main.py and the new service/router modules.

* refactor(arch): create api/deps + routers

- api/__init__.py, api/routers/__init__.py: package markers
- api/deps.py: verify_api_key, get_db, get_pipeline FastAPI dependencies
- api/routers/health.py: GET /healthz and GET / (index)
- api/routers/transcriptions.py: /api/transcribe, /api/jobs/*, /api/transcriptions/*, /api/export/*
- api/routers/voiceprints.py: all /api/voiceprints/* endpoints

No API behaviour changes; all URL, request, and response formats preserved.

* refactor(arch): extract services/

- services/__init__.py: package marker
- services/audio_service.py: convert_to_wav, compute_file_hash, lookup_hash,
  register_hash, _with_file_lock, _load_deepfilternet, _estimate_snr,
  maybe_denoise, and path-validation helpers (safe_tr_dir, safe_speaker_label,
  safe_log_filename)
- services/job_service.py: _LRUJobsDict, jobs singleton, _gpu_sem, and
  run_transcription worker (now accepts pipeline + voiceprint_db as args
  instead of relying on module-level globals)

* refactor(arch): slim main.py to ~60 lines

Replace 981-line monolith with a thin orchestration layer:
- asynccontextmanager lifespan handles startup (dirs, VoiceprintDB, cohort,
  TranscriptionPipeline) and stores objects on app.state
- CORS middleware and security-header / API-key middlewares retained verbatim
- Route registration via include_router for health, transcriptions, voiceprints
- All business logic moved to services/ and api/routers/

* perf(upload): PERF-C2 async file write + streaming SHA256, unblock event loop

Replace synchronous open()+file.file.read()+write() loop and the separate
compute_file_hash() second-pass read with a single save_upload_and_hash()
coroutine that uses aiofiles for non-blocking disk I/O and updates the SHA-256
digest on each chunk in-flight. Eliminates 2-10 s event-loop stall on large
(~500 MB) uploads; all other concurrent requests are no longer serialised
behind the upload handler.

* fix(jobs): AR-C2 write-through status.json, fallback read + orphan recovery on restart

Add _write_status() called at every jobs[job_id]["status"] mutation so each
state transition is atomically persisted to TRANSCRIPTIONS_DIR/{job_id}/status.json.
GET /api/jobs/{job_id} falls back to disk when the job is absent from the LRU dict,
returning completed+result, failed+error, or a synthetic failed response for stale
in-progress states. recover_orphan_jobs() is called in lifespan startup to rewrite
any pre-existing non-terminal status.json files to failed before the new process
begins accepting requests.

* test(core): TEST-C1~C4 security paths, VoiceprintDB logic, job persistence

- TEST-C1 (test_security.py): middleware 401s without creds, accepts
  Bearer and X-API-Key, tr_id regex rejects path traversal, enroll_speaker
  refuses pickle-laden .npy (SEC-C1 allow_pickle=False regression guard).
- TEST-C2 (test_voiceprint_db.py): real SQLite/sqlite-vec backed add+
  identify round trip, zero-vector defence, single-sample threshold
  relaxation to ~0.70, case-insensitive dedup-by-name, AS-norm only
  active when cohort >= 10, static-SQL update_speaker with/without name.
- TEST-C3 (test_job_service.py): run_transcription persists status.json
  on success, recover_orphan_jobs flips leftover in-progress jobs to
  failed, _LRUJobsDict evicts oldest entry past maxsize.
- TEST-C4: AS-norm active-flag regression folded into test_voiceprint_db
  plus a dedicated _effective_threshold pure-function pin.
- conftest: prefer real fastapi/voiceprint_db when installed; app_client
  fixture now chdir's to app/ (so StaticFiles finds ./static) and
  invalidates module caches so DATA_DIR is re-evaluated per test.

* fix(pre-release): C1 job_id路径验证, C2 CI测试门, I1-I6 import/status/Content-Disposition/cohort_size

[C1] GET /api/jobs/{job_id}: job_id 参数改用 FPath(pattern=...) 校验,与其他端点防护一致
[C2] CI: 移除 || true,只跑 test_security/test_voiceprint_db/test_job_service,补齐 aiofiles 等依赖
[I1] _write_status: 新增可选 filename 参数,converting 阶段写入 audio_path.name
[I2] list_transcriptions: 遍历时加 is_dir() 过滤,跳过非目录条目
[I3] voiceprints.py: 移除未使用的 get_pipeline import
[I4] deps.py: 移除未使用的 ALLOW_NO_AUTH import
[I5] Content-Disposition: srt/txt 导出补 RFC 6266 双引号包裹文件名
[I6] VoiceprintDB: 新增 cohort_size property,job_service 改用公开接口替代 ._asnorm._cohort 访问

* docs: sync all docs to v0.6.0, add CONTRIBUTING.md, rewrite READMEs

- Rewrite README.md and README.en.md in openclaw style (centered badges,
  nav links, star history chart)
- Add CONTRIBUTING.md (bilingual zh/en)
- Add v0.6.0 changelog entries (zh + en)
- Fix API docs: job persistence fallback, enroll idempotency, 504 error
- Fix security docs: SQLite WAL (was stale tempfile+os.replace), add 6
  new hardening items, add ALLOW_NO_AUTH note
- Fix quickstart docs: add JOBS_MAX_CACHE, FFMPEG_TIMEOUT_SEC, ALLOW_NO_AUTH
- Fix Critical #1: remove router-level verify_api_key dep (Bearer auth 403)
- Fix Critical #2: catch corrupt result.json in list_transcriptions
- Add .full-review/ to .gitignore

* fix: correct pipeline init arg order and pyannote token kwarg

- main.py: TranscriptionPipeline(model, DEVICE, HF_TOKEN) — was
  swapped to (model, HF_TOKEN, DEVICE), silently mapping HF token
  string to the device argument (torch rejected it as an invalid device)
- pipeline.py: use use_auth_token= for pyannote from_pretrained() calls;
  pyannote.audio==3.1.1 with huggingface_hub<0.24 exposes use_auth_token,
  not the newer token= kwarg

* test(e2e): add core API E2E test suite

37 tests covering auth (X-API-Key + Bearer), health, transcription
lifecycle, schema, error handling, voiceprints, and export.
Replaces the old OSD/separation A/B tests which referenced removed features.
Run: VOSCRIPT_URL=https://nas.esazx.com:8780 VOSCRIPT_KEY=... pytest tests/e2e/test_api_core.py

* feat: cleanup converted audio artifacts + add audio download endpoint

- job_service: delete intermediate .wav (convert) and .denoised.wav
  (denoise) after transcription completes; only original uploaded file
  is kept. Applies on both success and failure paths.
- transcriptions router: add GET /api/transcriptions/{tr_id}/audio
  to serve the original uploaded file for download
- e2e tests: add TestDedup (same-file dedup) and TestArtifactCleanup
  (audio download endpoint) test classes

* feat: v0.7.0 — speaker consolidation + no_repeat_ngram_size

Bug fix:
- Speaker cluster consolidation: multiple diarization clusters matching
  the same enrolled speaker are now merged to a single canonical label
  (highest similarity wins) before segment assembly. Previously the same
  person could appear as separate speakers in the result.

Features:
- no_repeat_ngram_size API parameter (default 0 = off, effective only
  when >= 3). Suppresses n-gram repetitions in faster-whisper output.
  Non-integer values return HTTP 422.
- params object now records no_repeat_ngram_size in completed job results.

Tests:
- 43 new E2E tests across 8 classes (TestSpeakerConsolidation, TestSecurity,
  TestSegmentReassignment, TestSpeakerManagement, TestExportFormats,
  TestOutputSchema, TestNoRepeatNgramSize, TestEdgeCases, TestLongChains).
- Suite: 84 collected, 78 pass, 6 expected skips.
- Add bench_overlap.py for overlap statistics benchmarking.

Docs:
- Fix MIT -> Apache 2.0 in README badges and license sections.
- Add no_repeat_ngram_size to API reference (zh + en).
- Add v0.7.0 changelog entries (zh + en).
- Align app version to 0.7.0 (was incorrectly set to 1.0.0).

Deployment:
- docker-compose.yml adds ./app:/app volume mount for hot-reload without
  image rebuilds.

* docs: rewrite READMEs + fix Voice Transcribe → VoScript

- Centered header: title, language switcher, badges, tagline, nav links
  all inside <div align=center>
- Rewrite opening description in direct voice: what VoScript is,
  what problem it solves (speaker labeling every recording by hand),
  why it matters (enroll once, recognized forever)
- Remove hanging blockquote citation for OpenPlaud
- Tighten feature list (remove implementation detail noise)
- app/main.py: FastAPI title 'Voice Transcribe' → 'VoScript'

* docs: rename OpenPlaud(Maple) → BetterAINote across all docs

* ci: publish Docker image to Docker Hub on version tag

* fix: remove unused imports (ruff F401)
@dependabot @github

dependabot Bot commented on behalf of github Apr 21, 2026

Copy link
Copy Markdown
Author

Looks like torch is no longer a dependency, so this is no longer needed.

@dependabot dependabot Bot closed this Apr 21, 2026
@dependabot dependabot Bot deleted the dependabot/pip/app/torch-2.8.0 branch April 21, 2026 10:53
MapleEve added a commit that referenced this pull request Apr 21, 2026
* feat: add overlapped speech detection (OSD) + MossFormer2 separation

Method A — OSD only (pyannote OverlappedSpeechDetection):
- TranscriptionPipeline.detect_overlaps(audio_path, onset) → {intervals, total_s, overlap_s, ratio, count}
- TranscriptionPipeline.osd_pipeline lazy property (onset-cache-aware)
- /api/transcribe accepts osd=true, osd_onset=0.08
- Each segment gets has_overlap: bool
- Result gets overlap_stats: {ratio, overlap_s, total_s, count}

Method B — OSD + MossFormer2 (clearvoice):
- TranscriptionPipeline.separate_overlaps(audio_path, n_speakers=2) → list[str]
- /api/transcribe accepts separate_speech=true (only runs when osd=true and ratio>0)
- Result gets separated_tracks: [{track, segments}]

Frontend:
- OSD checkbox in upload options
- ⚡ 重叠 amber badge on overlapping segments
- Overlap stats summary banner above segment list

* Revert "feat: add overlapped speech detection (OSD) + MossFormer2 separation"

This reverts commit ac4db1c.

* fix(pipeline): MIN_EMBED_DURATION 改为1.5s,支持env配置

- [CQ-H1] WeSpeaker ResNet34 推荐 ≥1.5s 输入;提高最短 turn 阈值并新增 MIN_EMBED_DURATION / MAX_EMBED_DURATION env 支持
- 超过 MAX_EMBED_DURATION 的 chunk 自动截断,避免显存浪费
- [BP-H6] use_auth_token 已废弃,改为 token(huggingface_hub >= 0.17)
- [BP-L3] 整理 Path import 到文件顶部,保留 GPU 库的 lazy import

* fix(security): 修复 XSS - escapeJs 完整转义 + data-* 事件委托

- 删除不完整的 escapeJs(仅转义 '/\),统一使用 escapeHtml 覆盖 &"'<>
- 动态生成的按钮改用 data-action/data-* 属性 + 全局事件委托,
  消除用户数据拼入 inline onclick 字符串的 XSS 通道
  (enrollSpeaker / deleteVp / renameVp / loadTranscription)
- 新增 CSP meta 标签作为服务端响应头的 fallback 防线

* fix(dockerfile): 删除冗余chown,添加digest pin TODO

- BP-L4: 删除 COPY --chown 之后冗余的 RUN chown -R app:app /app
- CD-H6 / BP-M6: 为基础镜像添加 digest pin TODO 注释
- BP-L5: 为 git 依赖添加 fake_git hack 清理后可移除的 TODO

* fix(security): ffmpeg 添加超时保护,删除 /tmp/_fake_git 死代码

- _convert_to_wav 增加 FFMPEG_TIMEOUT_SEC(默认 1800s)超时,
  防止畸形音频使 ffmpeg 死循环占满 CPU/内存、挂起 GPU 信号量;
  TimeoutExpired 时清理残留 wav 并返回 504
- 删除 _load_deepfilternet 中 /tmp/_fake_git shim 与 PATH prepend,
  容器 Dockerfile 已安装真实 git,该 world-writable 回退路径为
  死代码且构成 CWE-427 不可信搜索路径风险

* fix(voiceprint): identify()零向量防御,use_auth_token改token

- [CQ-H9] identify() 入口拒绝零向量 query,返回 (None, None, 0.0),避免 AS-norm 与 raw cosine 语义不一致
- [BP-H9] update_speaker 动态 SQL f-string 拆为两条静态 SQL(有/无 name 两路径)
- [BP-M7] Optional[T] 类型注解全量改为 T | None(Python 3.10+ 风格)
- [CQ-M12] build_cohort_from_transcriptions 硬编码 shape==(256,) 改为 EMBEDDING_DIM env 可配
- [CQ-M3] build_cohort 的 except Exception 改为 logger.warning,不再吞异常
- [BP-L3] 函数体内 base64/glob/os import 移到文件顶部

* fix(security): 添加安全响应头中间件,修复日志注入

- 新增 add_security_headers middleware,默认注入
  X-Content-Type-Options / X-Frame-Options / Referrer-Policy /
  X-XSS-Protection,与 CSP meta 配合构成最小安全头集
- 新增 _safe_log_filename 去除控制字符(\\x00-\\x1f/\\x7f),
  在上传路径用它清洗 file.filename,避免换行/ANSI 控制符
  原样写入日志造成 CWE-117 日志注入

* fix(main): reassign_speaker同步speaker_map

- [CQ-H7] reassign_speaker 更新 segment 时同步更新 speaker_map 中对应 label 的 matched_name/matched_id,保证人工纠错传导
- [CQ-H6] 空 embeddings 时写入 result.json 的 warning="no_speakers_detected" 标志,前端可区分"识别失败"与"无可用 speaker"并避免传 'undefined' 字符串

* fix(frontend): 轮询改指数退避,fetch 错误提示

- pollStatus 从固定 2000ms setInterval 改为 setTimeout 指数退避
  (×1.5,封顶 10000ms),减少长任务轮询压力
- loadHistory / loadTranscription / loadVoiceprints / enrollSpeaker /
  renameVp / deleteVp 外层补 try/catch,非 2xx 响应与网络错误通过
  toast 向用户提示状态码,不再静默失败(avoid 401/413 重复提示,
  由 apiFetch 统一处理)

* fix(docs): 修正AS-norm冷启动、阈值、cohort生命周期、OSD删除、enroll幂等性、jobs/transcriptions双状态

- DOC-H1: 修正 AS-norm 冷启动行为描述(cohort=0 / <10 / >=10 三档阈值与路径)
- DOC-H3: 阈值描述改为自适应(单样本~0.70,spread放宽,绝对下限0.60)
- DOC-H5: 新增 AS-norm cohort 生命周期章节(启动构建、任务不刷新、rebuild-cohort 手动触发)
- DOC-H4: 标注 OSD 功能已在 v0.5.x 中移除(changelog/benchmarks 加历史注释)
- DOC-H2: enroll 端点添加幂等性警告(不传 speaker_id 会产生重复记录)
- DOC-H6: /api/jobs 说明进程重启后返回 404,持久化结果改用 /api/transcriptions
- DOC-M1: /api/transcriptions/{id} 补充 speaker_map 与 unique_speakers 字段说明
- DOC-M5: 0.5.0 升级迁移说明,历史转录需 rebuild-cohort 才能激活 AS-norm

* feat(ci): 添加基础CI workflow + docker-compose healthcheck

- CD-C1: 新增 .github/workflows/ci.yml 做基础 lint (ruff) + pytest 守门
  - lint: ruff check + ruff format --check,忽略 E501
  - test: 轻量依赖跑单元测试,排除 e2e/separation;暂用 || true 避开针对已删除
    OSD 功能的测试,待测试套件修复后改为强制失败
- CD-M2: docker-compose.yml 为 voscript 服务加 healthcheck,使用 /healthz 探测,
  start_period=120s 覆盖首次模型加载时间

* fix(style): CORS headers 收敛,__import__ 改标准 import

- threading 提升到模块顶部 import,删除函数体内的
  import threading as _threading 和 __import__("threading") 丑陋用法,
  _gpu_sem / _hash_index_lock 改用 threading.Lock/Semaphore
- CORSMiddleware 的 allow_headers 从通配 ["*"] 收敛为显式白名单:
  Authorization / Content-Type / X-API-Key / X-Request-Id,
  符合最小权限原则

* fix(quality): 静态SQL替换f-string,异常日志改善

- [CQ-M13] _format_srt_time / _format_timestamp 防御 None / NaN / 负秒,避免 int() 抛异常
- [CQ-M3] _run_transcription 中两处 CUDA 缓存 flush 的 except Exception: pass 改为 logger.warning
- [CQ-M10] rebuild-cohort 端点返回 skipped 字段,通过 voiceprint_db.last_cohort_skipped 暴露跳过/损坏文件数

* fix(perf+docs): PERF-H8 batch cosine scan, DOC-C1 similarity semantics, DOC-C2 API_KEY warning, CD-C2 pip-audit CI

* fix(security): SEC-C1 pickle RCE, SEC-C2 path traversal, SEC-C3 API_KEY warning, BP-C2 path param validation, CD-C3 daemon thread

- SEC-C1 (CVSS 9.1): np.load(emb_path) → np.load(emb_path, allow_pickle=False)
  in enroll_speaker to prevent pickle-based RCE via crafted .npy files.

- SEC-C2 (CVSS 9.6): Add _safe_tr_dir(tr_id) and _safe_speaker_label(label)
  validators. All route handlers that touch TRANSCRIPTIONS_DIR / tr_id now
  call _safe_tr_dir(); enroll_speaker validates speaker_label before building
  the embedding filename. Both use allowlist regex + .resolve() confinement.

- SEC-C3 (CVSS 9.8): Add ALLOW_NO_AUTH env var. When API_KEY is None and
  ALLOW_NO_AUTH != "1", emit a highly-visible WARNING so operators cannot
  accidentally ship the service open. Service still starts (warning not fatal).

- BP-C2: All tr_id path parameters now carry an Annotated[str, FPath(pattern=...)]
  constraint so FastAPI validates the format at the framework layer before any
  handler code runs (get_transcription, reassign_speaker, export_transcription).

- CD-C3: Thread(..., daemon=True) on the transcription worker so SIGTERM
  causes the process to exit cleanly rather than hanging on in-flight jobs.

* fix(logic): CQ-C1 cohort auto-rebuild, CQ-C2 AS-norm threshold, CQ-C3 add_speaker dedup, CQ-C4 vec serialize, CQ-H2 LRU jobs, CQ-H5 fcntl lock

CQ-C1: _run_transcription now increments a counter after each successful job
and rebuilds the AS-norm cohort when absent or every 10th transcription, so
normalization activates without a server restart.

CQ-C2: identify() no longer sets asnorm_active=True when ASNormScorer.score()
fell back to raw cosine (cohort < 10). Score is only treated as AS-norm
normalized — and compared against the 0.5 operating threshold — when the
cohort has >= 10 embeddings; otherwise the adaptive per-speaker threshold
logic takes over.

CQ-C3: add_speaker() now performs a case-insensitive name lookup before
INSERT. Duplicate names route to update_speaker() (append sample + recompute
average) instead of creating a second record.

CQ-C4: The sqlite-vec MATCH query now serialises the query vector via
sqlite_vec.serialize_float32() when available, falling back to
struct.pack('<Nf', ...) for explicit little-endian byte order. Replaces
query.tobytes() which is platform byte-order-dependent.

CQ-H2/PERF-C1: The unbounded `jobs: dict` replaced with _LRUJobsDict
(OrderedDict-backed, thread-safe, bounded by JOBS_MAX_CACHE env var,
default 200). Prevents memory growth from accumulating full result payloads
indefinitely.

CQ-H5: _hash_index_lock (threading.Lock, single-process only) replaced with
_with_file_lock() using fcntl.flock(LOCK_EX) for cross-process mutual
exclusion when running uvicorn --workers > 1. Falls back to thread lock on
non-Unix platforms.

* perf(pipeline): PERF-H1 segment-level torchaudio.load, avoid whole-file OOM

Replace the single torchaudio.load(audio_path) call that decoded the
entire WAV file into memory with per-segment loads using the
frame_offset/num_frames API (torchaudio >= 0.9).  File metadata is
obtained via torchaudio.info() which reads only the header.  Each
iteration allocates at most MAX_EMBED_DURATION * sr * 4 bytes (~640 KB
at 16 kHz / 10 s) instead of the full file, eliminating OOM on 2-hour
recordings (~900 MB–2 GB WAV).  Resampling and mono downmix are applied
per chunk; a try/except guard logs and skips unreadable segments without
aborting the pipeline.

* refactor(arch): extract config.py

Centralise all os.getenv() calls into app/config.py so environment-variable
names are defined in one auditable place. Removes scattered getenv() calls
from main.py and the new service/router modules.

* refactor(arch): create api/deps + routers

- api/__init__.py, api/routers/__init__.py: package markers
- api/deps.py: verify_api_key, get_db, get_pipeline FastAPI dependencies
- api/routers/health.py: GET /healthz and GET / (index)
- api/routers/transcriptions.py: /api/transcribe, /api/jobs/*, /api/transcriptions/*, /api/export/*
- api/routers/voiceprints.py: all /api/voiceprints/* endpoints

No API behaviour changes; all URL, request, and response formats preserved.

* refactor(arch): extract services/

- services/__init__.py: package marker
- services/audio_service.py: convert_to_wav, compute_file_hash, lookup_hash,
  register_hash, _with_file_lock, _load_deepfilternet, _estimate_snr,
  maybe_denoise, and path-validation helpers (safe_tr_dir, safe_speaker_label,
  safe_log_filename)
- services/job_service.py: _LRUJobsDict, jobs singleton, _gpu_sem, and
  run_transcription worker (now accepts pipeline + voiceprint_db as args
  instead of relying on module-level globals)

* refactor(arch): slim main.py to ~60 lines

Replace 981-line monolith with a thin orchestration layer:
- asynccontextmanager lifespan handles startup (dirs, VoiceprintDB, cohort,
  TranscriptionPipeline) and stores objects on app.state
- CORS middleware and security-header / API-key middlewares retained verbatim
- Route registration via include_router for health, transcriptions, voiceprints
- All business logic moved to services/ and api/routers/

* perf(upload): PERF-C2 async file write + streaming SHA256, unblock event loop

Replace synchronous open()+file.file.read()+write() loop and the separate
compute_file_hash() second-pass read with a single save_upload_and_hash()
coroutine that uses aiofiles for non-blocking disk I/O and updates the SHA-256
digest on each chunk in-flight. Eliminates 2-10 s event-loop stall on large
(~500 MB) uploads; all other concurrent requests are no longer serialised
behind the upload handler.

* fix(jobs): AR-C2 write-through status.json, fallback read + orphan recovery on restart

Add _write_status() called at every jobs[job_id]["status"] mutation so each
state transition is atomically persisted to TRANSCRIPTIONS_DIR/{job_id}/status.json.
GET /api/jobs/{job_id} falls back to disk when the job is absent from the LRU dict,
returning completed+result, failed+error, or a synthetic failed response for stale
in-progress states. recover_orphan_jobs() is called in lifespan startup to rewrite
any pre-existing non-terminal status.json files to failed before the new process
begins accepting requests.

* test(core): TEST-C1~C4 security paths, VoiceprintDB logic, job persistence

- TEST-C1 (test_security.py): middleware 401s without creds, accepts
  Bearer and X-API-Key, tr_id regex rejects path traversal, enroll_speaker
  refuses pickle-laden .npy (SEC-C1 allow_pickle=False regression guard).
- TEST-C2 (test_voiceprint_db.py): real SQLite/sqlite-vec backed add+
  identify round trip, zero-vector defence, single-sample threshold
  relaxation to ~0.70, case-insensitive dedup-by-name, AS-norm only
  active when cohort >= 10, static-SQL update_speaker with/without name.
- TEST-C3 (test_job_service.py): run_transcription persists status.json
  on success, recover_orphan_jobs flips leftover in-progress jobs to
  failed, _LRUJobsDict evicts oldest entry past maxsize.
- TEST-C4: AS-norm active-flag regression folded into test_voiceprint_db
  plus a dedicated _effective_threshold pure-function pin.
- conftest: prefer real fastapi/voiceprint_db when installed; app_client
  fixture now chdir's to app/ (so StaticFiles finds ./static) and
  invalidates module caches so DATA_DIR is re-evaluated per test.

* fix(pre-release): C1 job_id路径验证, C2 CI测试门, I1-I6 import/status/Content-Disposition/cohort_size

[C1] GET /api/jobs/{job_id}: job_id 参数改用 FPath(pattern=...) 校验,与其他端点防护一致
[C2] CI: 移除 || true,只跑 test_security/test_voiceprint_db/test_job_service,补齐 aiofiles 等依赖
[I1] _write_status: 新增可选 filename 参数,converting 阶段写入 audio_path.name
[I2] list_transcriptions: 遍历时加 is_dir() 过滤,跳过非目录条目
[I3] voiceprints.py: 移除未使用的 get_pipeline import
[I4] deps.py: 移除未使用的 ALLOW_NO_AUTH import
[I5] Content-Disposition: srt/txt 导出补 RFC 6266 双引号包裹文件名
[I6] VoiceprintDB: 新增 cohort_size property,job_service 改用公开接口替代 ._asnorm._cohort 访问

* docs: sync all docs to v0.6.0, add CONTRIBUTING.md, rewrite READMEs

- Rewrite README.md and README.en.md in openclaw style (centered badges,
  nav links, star history chart)
- Add CONTRIBUTING.md (bilingual zh/en)
- Add v0.6.0 changelog entries (zh + en)
- Fix API docs: job persistence fallback, enroll idempotency, 504 error
- Fix security docs: SQLite WAL (was stale tempfile+os.replace), add 6
  new hardening items, add ALLOW_NO_AUTH note
- Fix quickstart docs: add JOBS_MAX_CACHE, FFMPEG_TIMEOUT_SEC, ALLOW_NO_AUTH
- Fix Critical #1: remove router-level verify_api_key dep (Bearer auth 403)
- Fix Critical #2: catch corrupt result.json in list_transcriptions
- Add .full-review/ to .gitignore

* fix: correct pipeline init arg order and pyannote token kwarg

- main.py: TranscriptionPipeline(model, DEVICE, HF_TOKEN) — was
  swapped to (model, HF_TOKEN, DEVICE), silently mapping HF token
  string to the device argument (torch rejected it as an invalid device)
- pipeline.py: use use_auth_token= for pyannote from_pretrained() calls;
  pyannote.audio==3.1.1 with huggingface_hub<0.24 exposes use_auth_token,
  not the newer token= kwarg

* test(e2e): add core API E2E test suite

37 tests covering auth (X-API-Key + Bearer), health, transcription
lifecycle, schema, error handling, voiceprints, and export.
Replaces the old OSD/separation A/B tests which referenced removed features.
Run: VOSCRIPT_URL=https://nas.esazx.com:8780 VOSCRIPT_KEY=... pytest tests/e2e/test_api_core.py

* feat: cleanup converted audio artifacts + add audio download endpoint

- job_service: delete intermediate .wav (convert) and .denoised.wav
  (denoise) after transcription completes; only original uploaded file
  is kept. Applies on both success and failure paths.
- transcriptions router: add GET /api/transcriptions/{tr_id}/audio
  to serve the original uploaded file for download
- e2e tests: add TestDedup (same-file dedup) and TestArtifactCleanup
  (audio download endpoint) test classes

* feat: v0.7.0 — speaker consolidation + no_repeat_ngram_size

Bug fix:
- Speaker cluster consolidation: multiple diarization clusters matching
  the same enrolled speaker are now merged to a single canonical label
  (highest similarity wins) before segment assembly. Previously the same
  person could appear as separate speakers in the result.

Features:
- no_repeat_ngram_size API parameter (default 0 = off, effective only
  when >= 3). Suppresses n-gram repetitions in faster-whisper output.
  Non-integer values return HTTP 422.
- params object now records no_repeat_ngram_size in completed job results.

Tests:
- 43 new E2E tests across 8 classes (TestSpeakerConsolidation, TestSecurity,
  TestSegmentReassignment, TestSpeakerManagement, TestExportFormats,
  TestOutputSchema, TestNoRepeatNgramSize, TestEdgeCases, TestLongChains).
- Suite: 84 collected, 78 pass, 6 expected skips.
- Add bench_overlap.py for overlap statistics benchmarking.

Docs:
- Fix MIT -> Apache 2.0 in README badges and license sections.
- Add no_repeat_ngram_size to API reference (zh + en).
- Add v0.7.0 changelog entries (zh + en).
- Align app version to 0.7.0 (was incorrectly set to 1.0.0).

Deployment:
- docker-compose.yml adds ./app:/app volume mount for hot-reload without
  image rebuilds.

* docs: rewrite READMEs + fix Voice Transcribe → VoScript

- Centered header: title, language switcher, badges, tagline, nav links
  all inside <div align=center>
- Rewrite opening description in direct voice: what VoScript is,
  what problem it solves (speaker labeling every recording by hand),
  why it matters (enroll once, recognized forever)
- Remove hanging blockquote citation for OpenPlaud
- Tighten feature list (remove implementation detail noise)
- app/main.py: FastAPI title 'Voice Transcribe' → 'VoScript'

* docs: rename OpenPlaud(Maple) → BetterAINote across all docs

* ci: publish Docker image to Docker Hub on version tag

* fix: remove unused imports (ruff F401)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants